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ABSTRACT 


The  purpose  of  this  literature  review  is  to  provide  the  background  to  an  evaluation 
of  the  utility  of  the  Command  Safety  Assessment  Survey  (CSAS)  as  a  valid  predictor  of 
future  mishaps.  The  end  goal  is  to  be  able  to  use  the  survey  to  identify  “at  risk”  U.S. 
Naval  squadrons  prior  to  the  occurrence  of  mishaps.  Safety  climate  describes  employees’ 
perceptions,  attitudes,  and  beliefs  about  risk  and  safety  (Mearns  &  Flin,  1999).  Safety 
climate  is  most  commonly  evaluated  using  questionnaires.  Although  assessments  of 
safety  climate  are  not  widespread  in  civil  aviation,  the  United  States  Navy  has  been  using 
the  CSAS  since  2000  to  measure  the  safety  climate  of  aviation  squadrons.  This  review 
argues  that  a  comprehensive  assessment  of  the  construct  (the  extent  to  which  the 
questionnaire  measures  what  it  is  intended  to  measure)  and  discriminate  validity 
(correlate  the  data  from  the  questionnaire  with  a  criterion  variable,  such  as  accidents)  of 
the  CSAS  should  be  carried  out.  This  assessment  is  necessary  to  ensure  that  squadron 
Commanding  Officers,  and  senior  leadership,  are  being  provided  with  valid  and  reliable 
information  on  squadron  safety  climate. 
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I.  INTRODUCTION 


The  purpose  of  this  literature  review  is  to  provide  the  background  to  an  evaluation  of  the 
utility  of  the  Command  Safety  Assessment  Survey  (CSAS)  as  a  valid  predictor  of  future 
mishaps.  The  end  goal  is  to  be  able  to  use  the  survey  to  identify  “at  risk”  U.S.  Naval  squadrons 
prior  to  the  occurrence  of  mishaps.  The  CSAS  was  designed  to  measure  the  safety  climate  of 
U.S.  Naval  aviation  squadrons.  In  this  literature  review,  safety  climate  will  be  defined  and  the 
method  of  measurement  outlined.  The  literature  concerning  the  correlation  of  safety  climate  with 
other  indicators  of  safety  perfonnance  will  be  discussed.  Finally,  the  research  on  safety  climate 
that  has  been  carried  out  in  aviation  will  be  delineated,  with  a  specific  emphasis  on  the  method 
used  to  assess  the  safety  climate  in  U.S.  military  aviation. 

The  military  operates  in  a  high-risk  environment,  utilizing  highly  complex  technologies 
to  achieve  mission  goals.  The  reliability  of  the  hardware  and  software  of  these  complex  systems 
has  been  steadily  improving,  resulting  in  dramatic  decreases  in  the  number  of  failures  over  the 
last  century  (O’Connor  &  Cohen,  2010).  To  illustrate,  in  U.S  Naval  aviation,  776  aircraft  were 
destroyed  due  to  accidents  in  1954,  compared  to  only  24  in  2000  (Wiegmann  &  Shappell,  2003). 
However,  although  the  absolute  mishap  rate  has  decreased,  the  proportion  of  mishaps  attributed 
to  human  error  has  not  decreased  at  the  same  rate  as  the  mishaps  involving  mechanical  and 
environmental  factors  (Wiegmann  &  Shappell,  2003).  In  U.S.  Naval  aviation,  human  error 
accounts  for  more  than  80%  of  mishaps  (Naval  Safety  Center,  2006).  This  finding  is  not  unique 
to  U.S.  Naval  aviation,  as  between  80%  and  90%  of  all  work-related  accidents  and  incidents  can 
be  attributed  to  human  error  (Health  and  Safety  Executive,  2002;  Hollnagel,  1993;  Reason, 
1990).  Therefore,  as  has  been  the  case  with  other  High  Reliability  Organizations  (HROs;  those 
organizations  which  are  operating  technology  that  is  sufficiently  complex  to  be  subject  to 
catastrophic  accidents;  Shrivastava,  1986),  the  United  States  military  has  recognized  the  need  to 
focus  upon  the  human  causes  of  mishaps. 

Traditionally,  safety  performance  has  been  assessed  solely  on  the  basis  of  “lagging 
indicators”  of  safety  such  as  fatalities,  or  mishap  rates.  However,  as  safety  has  improved  and  the 
frequency  of  mishaps  has  declined,  mishap  rates  have  ceased  to  be  a  useful  metric  of  safety 
performance.  Therefore,  HROs  have  started  to  examine  “leading  indicators”  of  safety.  The 
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United  Kingdom  Health  and  Safety  Executive  (HSE,  2006)  defined  leading  indicators  of  safety 
as  measures  of  process  or  inputs  essential  to  deliver  the  desired  safety  outcomes  (e.g.,  safety 
climate  surveys,  hazard  reports).  Lagging  indicators  show  when  a  desired  safety  outcome  has 
failed  or  has  not  been  achieved  (e.g.,  number  of  mishaps).  Therefore,  leading  indicators  of  safety 
are  used  in  an  attempt  to  gain  insight  into  the  safety  perfonnance  of  the  organization  and  identify 
areas  in  which  efforts  should  be  made  to  improve  safety. 

A.  DEFINITIONS  OF  SAFETY  CULTURE  AND  SAFETY  CLIMATE 

Zohar  (1980)  defined  safety  climate  as  a  summary  of  perceptions  that  employees  share 
about  their  work  environment.  Safety  climate  describes  employees’  perceptions,  attitudes,  and 
beliefs  about  risk  and  safety  (Mearns  &  Flin,  1999).  It  is  a  “snapshot”  of  the  current  state  of 
safety  in  the  organization.  There  has  been  an  ongoing  debate  within  the  literature  regarding  the 
use  of  the  terms  “culture”  and  “climate,”  and  whether  they  represent  the  same  or  different 
concepts.  The  general  consensus  is  that  culture  represents  the  more  stable  and  enduring 
characteristics  of  the  organization,  and  has  been  likened  to  its  traits  or  “personality.”  Safety 
culture  is  a  more  complex  and  enduring  trait,  reflecting  fundamental  values,  norms,  assumptions, 
and  expectations,  which,  to  some  extent,  reside  in  societal  culture  (Mearns  &  Flin,  1999). 
Climate,  on  the  other  hand,  is  thought  to  represent  a  more  visible  manifestation  of  the  culture, 
which  can  be  seen  as  its  “mood  state,”  at  a  particular  moment  in  time  (Cox  &  Flin,  1998). 

Denison  (1996)  argues  that  the  methods  used  by  researchers  can  help  to  distinguish 
between  culture  and  climate  studies.  He  argues  that  culture  requires  qualitative  measures,  while 
climate  requires  quantitative  measures.  Because  the  questionnaire  survey  is  the  predominant 
method  used  for  investigating  safety,  it  is  now  widely  recognized  that  this  method  reflects  the 
climate  of  the  organization  at  the  time  of  the  study  (Denison,  1996).  However,  it  is  generally 
agreed  that  climate  can  be  used  as  an  indication  of  the  underlying  safety  culture  (Cox  &  Cheyne, 
2000;  Mearns  &  Flin,  1999).  The  point  is  put  succinctly  by  Rousseau  (1985),  who  states  that  the 
similarities  between  the  concepts  of  climate  and  culture  are  sufficiently  overlapping  for  research 
on  one  to  inform  us  about  the  other.  For  the  remainder  of  this  literature  review,  we  focus  on 
safety  climate. 
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B. 


MEASURING  SAFETY  CLIMATE 


As  discussed  above,  safety  climate  is  almost  predominately  measured  using  a 
questionnaire  methodology.  Guldenmund  (2007)  describes  this  method  as  a  quick,  but  also 
“dirty”  technique  for  measuring  safety  climate.  It  is  dirty  because  it  arguably  only  gives  a  little 
insight  into  the  safety  climate  of  the  organization  from  a  single  perspective.  Guldenmund  (2007) 
states  that  “the  challenge  is  to  develop  a  questionnaire  that  yields  just  enough  relevant 
information — the  trusted  ‘wet  finger’  to  find  out  which  way  the  wind  blows — to  decide  whether 
and  possibly  where  any  corrective  measures  or  actions  are  opportune”  (p.  724). 

Unlike  the  field  of  personality  assessment,  in  which  consensus  has  been  largely  reached 
regarding  personality  constructs,  there  has  been  no  such  agreement  regarding  safety  climate 
constructs.  It  is  debatable  whether  safety  climate  instruments  should  be  generic  or  specific  in 
nature  (Cox  &  Flin,  1998).  Cheyne,  Tomas,  Cox,  and  Oliver  (1999)  argued  that  the  architecture 
of  employee  attitudes  to  safety  was  context-dependent  and  varied  by  industrial  sector.  Likewise, 
Coyle,  Sleeman,  and  Adams  (1995)  found  different  factor  structures,  using  the  same  safety 
climate  scale,  in  two  Australian  health  care  organisations,  concluding  that  the  likelihood  of 
establishing  a  universal  and  stable  set  of  safety  climate  factors  was  highly  doubtful.  Zohar  (2003) 
concurs,  arguing  that  safety  climate  indicators  should  be  subdivided  into  universal  and  industry- 
specific  indicators. 

Over  40  different  safety  climate  measures  have  been  developed  (Yule,  O’Connor,  &  Flin, 
2003).  These  questionnaires  tend  to  be  self-administered,  and  can  be  delivered  to  a  large  number 
of  people  in  an  organization  relatively  easily.  The  first  stage  in  developing  a  safety  climate 
questionnaire  is  to  identify  a  number  of  thematic  items  that  are  thought  to  be  relevant  to  the 
safety  climate.  Guldenmund  (2007)  differentiates  between  two  methods  for  identifying  the  items: 
a  nonnative,  or  theoretical,  approach  in  which  the  items  are  derived  on  the  basis  of  a  theoretical 
model  of  safety  climate,  or  a  pragmatic  approach  in  which  the  questionnaire  builds  upon 
previous  research.  Responses  to  each  item  are  generally  assessed  using  a  Likert  scale.  For 
analytic  purposes,  these  scales  are  generally  considered  to  be  interval  (although  they  almost 
certainly  are  not),  so  that  multivariate  statistical  methods  can  be  used. 

The  items  are  designed  to  assess  a  particular  safety  climate  theme  (e.g.,  safety  systems). 

The  purpose  is  to  develop  a  number  of  scales  that  can  be  used  to  evaluate  whether  there  are 
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differences  between  groups  of  respondents  on  particular  aspects  of  the  safety  climate.  Using 
scales,  as  opposed  to  examining  the  responses  to  single  items,  allows  the  researcher  to  have  a 
greater  reliability  in  the  participant’s  view  of  a  particular  aspect  of  the  safety  climate. 

Once  the  data  has  been  collected,  exploratory  factor  analysis  is  used  to  identify  whether 
the  items  are  grouping  (or  loading)  on  the  themes  as  anticipated.  As  part  of  this  process,  items 
are  often  discarded.  Themes  also  may  be  deleted,  combined,  or  renamed.  This  adaption  to  the 
questionnaire  is  a  nonnal  part  of  the  factor  analysis  process.  Once  a  stable  factor  structure  has 
been  established,  attempts  may  then  be  made  to  confirm  this  structure  with  a  different  data  set. 
The  exploratory  and/or  confirmatory  factor  analyses  are  a  necessary  process  in  the  construction 
of  reliable  scales.  These  techniques  also  help  to  establish  the  construct  validity  of  the  tool. 
Construct  validity  is  concerned  with  the  extent  to  which  the  questionnaire  measures  what  it  is 
intended  to  measure.  Identification  of  a  reliable  factor  structure,  that  is  consistent  with  theory, 
helps  the  researcher  substantiate  claims  regarding  the  validity  of  the  questionnaire,  although 
there  is  no  consensus  on  the  specific  factors  that  comprise  the  safety  climate.  As  seen  in  a 
number  of  reviews  (e.g.,  Cohen,  1977;  Flin,  Mearns,  O’Connor,  &  Bryden,  2000;  Guldenmund, 
2000;  Hale  &  Hovden,  1998;  Shannon,  Mayr,  &  Haynes,  1997),  there  is  some  agreement 
regarding  the  themes  that  are  relevant  to  the  construct  of  safety  climate.  These  common  themes 
will  be  discussed  in  the  next  section. 

C.  COMMON  SAFETY  CLIMATE  THEMES 

Although  there  are  a  large  number  of  factors  that  have  been  identified  by  safety  climate 
researchers,  these  factors  can  be  reduced  to  a  limited  number  of  themes  (Gadd  &  Collins,  2002; 
Flin  et  ah,  2000).  To  illustrate,  in  a  review  of  18  safety  climate  questionnaires,  Flin  et  ah,  (2000) 
identified  six  common  themes:  management/supervision,  safety  systems,  risk,  work  pressure, 
competence/training,  and  procedures/rules.  Each  of  these  themes  will  be  discussed  below. 

1.  Management/Supervision 

A  factor  concerned  with  management  is  identified  about  75%  of  the  time  in  safety 
climate  research  (Gadd  &  Collins,  2002;  Flin  et  ah,  2000).  However,  this  term  is  rather  nebulous 
and  refers  to  a  wide  range  of  managerial  behaviors,  from  the  development  of  the  safety  program 
to  the  quality  of  labor-management  relations.  Nonetheless,  the  research  suggests  that  managers 
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can  demonstrate  their  commitment  to  safety  in  a  number  of  tangible  ways — first,  through  their 
commitment  to  structural  and  procedural  safety  systems  including  the  development  of  the  safety 
program.  This  program  includes  a  diverse  range  of  activities  such  as:  good  housekeeping  and 
environmental  conditions,  good  training  facilities,  clear  safety  policy  and  goals,  formal 
inspections  at  regular  and  frequent  intervals,  thorough  investigations  of  all  accidents  and  near 
misses,  thorough  record  keeping,  rules  and  regulations  regularly  updated  and  evidence  of 
management  and  staff  compliance  with  them,  a  high  priority  being  given  to  safety  at  company 
meetings,  an  active  safety  committee  and  a  high-ranking  safety  officer  (Cohen,  Smith,  &  Cohen, 
1975;  Smith,  Cohen,  Cohen,  &  Cleveland,  1978;  Simons  &  Shafai-Sharai,  1977).  Secondly,  and 
perhaps  more  importantly,  management  can  demonstrate  their  commitment  to  safety  through 
their  attitudes,  behaviors,  and  styles  of  leadership.  These  factors  tend  to  be  less  tangible  than  the 
structural  and  procedural  variables,  but  nonetheless  are  thought  to  have  at  least  as  powerful  an 
effect  on  workforce  safety  motivation  (Eyssen-McKeown,  Eakin,  Hoffmann,  &  Spengler,  1980; 
Andriessen,  1978;  Zohar,  1980,  2000).  In  fact,  Hale  and  Hovden  (1998)  suggest  that  structural 
factors  are  likely  to  be  critical  only  in  organizations  with  a  poor  safety  climate;  they  cease  to 
discriminate  once  the  company  has  achieved  a  modest  level  of  advancement  in  safety 
management.  It  is  the  other  less  tangible  factors  concerning  management’s  attitudes  and  style  of 
leadership,  as  well  as  the  nature  and  quality  of  interactions  with  the  workforce,  which  are  better 
at  discriminating  the  excellent  from  the  good  organizations. 

The  importance  of  interactions  between  managers  and  workers  has  been  clearly 
established  through  the  research  (Andriessen,  1978;  Hale  &  Hovden,  1998).  Specifically, 
management  participation  and  involvement  in  work  and  safety  activities,  as  well  as  frequent, 
informal  communications  between  workers  and  management,  are  recognized  as  critical 
behaviors.  These  interactions  serve  a  number  of  useful  functions:  they  demonstrate  the 
managers’  concern  for  safety;  serve  as  a  frame  of  reference  for  the  workforce  to  guide 
appropriate  task  behaviors;  foster  closer  ties  between  managers,  supervisors,  and  workers; 
encourage  a  free  exchange  of  ideas  on  job  improvement;  and  provide  an  opportunity  for  the  early 
recognition  of  hazards  and  improper  job  practices  (Cohen,  1977).  More  importantly, 
worker-management  interaction  provides  a  clear  indication  of  an  overt,  active,  and  genuine 
concern  for  safety  on  the  part  of  management.  The  evidence  strongly  supports  the  utility  of 
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management  demonstrating  the  priority  for  safety  over  production  goals  (Eyessen-McKeown 
et  al.,  1980;  Diaz  &  Cabrera,  1997).  Interestingly,  those  organizations  that  have  clear  safety 
goals  also  tend  to  be  more  productive  (Peters,  1989). 

A  decentralized  approach  to  safety  management  has  been  shown  to  be  the  most  effective 
way  in  which  management  can  promote  workforce  safety  motivation  (Griffin,  Burley,  &  Neal, 
2000;  Simard  &  Marchand,  1995).  This  approach  is  achieved  by  encouraging  the  joint 
involvement  of  supervisors  with  employees  in  relatively  structured  safety  activities.  Indeed, 
decentralized  management  at  all  levels  is  not  only  the  best  predictor  of  workgroups’  propensity 
to  safety  initiatives;  it  also  is  the  most  important  factor  in  relation  to  two  other  predictors  of 
worker  motivation  to  safety,  namely  workgroup  cooperation  and  cohesion  (Simard  &  Marchand, 

1995) .  Cooperative  relationships  are  characterized  by  a  positive  team  spirit  and  willingness  to 
cooperate  with  other  team  members,  and  other  teams,  in  order  to  achieve  the  organization’s 
goals.  Workgroups,  which  are  internally  cohesive  and  cooperative,  also  tend  to  be  more 
cooperative  with  management  (Griffin  et  al.,  2000).  Thus,  any  attempt  by  senior  management  to 
increase  workers’  safety  motivation  must  begin  by  attempting  to  increase  supervisors’  and 
workers’  capacity  to  behave  cooperatively  with  each  other,  thereby  meeting  their  social  and 
autonomous  needs. 

A  number  of  supervisory-level  variables  also  have  been  identified  as  being  associated 
with  good  safety  perfonnance.  Specifically,  participative  and  supportive  supervisory  behavior  is 
identified  as  critical  to  safe  perfonnance  (Niskanen,  1994;  Flin,  Mearns,  Gordon,  &  Fleming, 

1996) .  These  supervisory  behaviors  are  strongly  promoted  when  the  supervisors  are  allowed 
autonomy  within  their  own  jobs.  Numerous  studies  suggest  that  a  decentralized  approach  on  the 
part  of  more  senior  management  is  conducive  to  more  participative  relationships  further  down 
the  line  (Hofmann  &  Morgeson,  1999). 

The  empirical  evidence  suggests  that  it  is  not  just  management  commitment, 
participation,  and  involvement  in  safety  activities  that  are  important,  but  the  extent  to  which 
management  encourages  the  involvement  of  the  workforce.  In  particular,  the  workforce  must  be 
permitted  to  help  shape  interventions  rather  than  simply  be  passive  recipients.  In  this  way, 
workers  are  more  likely  to  take  ownership  and  responsibility  for  safety  and  to  become  actively 
motivated  to  take  personal  initiative  in  safety  (Niskanen,  1994;  Williamson,  Feyer,  Caims,  & 
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Biancotti,  1997).  Cohen  and  Cleveland  (1983)  make  the  following  observations  based  upon  their 
three-phased  study:  people  work  more  safely  when  they  are  involved  in  decision-making 
processes;  when  they  have  specific  and  reasonable  responsibilities;  authority  and  goals;  and 
when  they  have  immediate  feedback  about  their  job  performance. 

2.  Safety  Systems 

This  very  broad  theme  has  a  number  of  overlapping  dimensions  to  it,  including:  the 
condition  of  physical  plant  and  equipment;  safety  systems  such  as  Permit  to  Work  (PTW) 
systems,  hazard  identification  systems,  and  including  incident  reporting  systems;  accident 
investigation  and  record  keeping;  safety  rules,  policies,  and  procedures;  selection  promotion  and 
training;  safety  department  effectiveness;  and  communication  and  feedback  mechanisms 
(Flin  et  ah,  2000).  A  factor  related  to  safety  systems  is  identified  in  about  two-thirds  of  safety 
climate  studies  (Gadd  &  Collins,  2002;  Flin  et  al.,  2000). 

Perception  of  the  safety  systems  is  clearly  an  important  component  of  safety  climate.  For 
example,  in  a  safety  climate  study  of  offshore  oil  company  workers,  Mearns,  Flin,  Gordon,  and 
Fleming  (1998)  found  that  reporting  systems,  rules  and  procedures,  and  safety  systems  were 
among  the  key  factors  related  to  self-reported  accident  involvement.  Similarly,  in  an 
investigation  of  safety  practices  among  a  telephone  company’s  construction  and  maintenance 
workers,  Eyssen-McKeown  et  al.  (1980)  found  that  lower  injury  rates  were  associated  with  the 
perceived  effectiveness  of  safety  regulations  and  the  safety  program. 

Furthermore,  the  evidence  suggests  that  perceptions  of  leadership  commitment  to  safety 
appear  to  influence  opinions  about  the  safety  system,  which  in  turn  appear  to  influence 
employees’  at-risk  behaviors  and  injury  rates  (Simard  &  Marchand,  1994,  1995,  1997).  A 
comparison  of  high  and  low  accident  companies  has  shown  that  in  high-performing  companies, 
safety  policy  and  procedures  were  characterized  by  clarity,  consistency,  and  emphasis  (Gaertner, 
Newman,  Perry,  Fisher,  &  Whitehead,  1987). 

Unfortunately,  because  data  related  to  safety  systems  are  often  available  from  other  more 
traditional  methods  such  as  safety  audits  and  hazard  reports,  issues  related  to  the  safety  system 
are  often  omitted  from  climate  scales.  Bailey  and  Petersen  (1989)  concluded  that  the 
effectiveness  of  safety  programs  cannot  be  measured  by  the  more  traditional  procedural¬ 
engineering  criteria  popularly  thought  to  be  factors  in  successful  programs.  They  argue  that  a 
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better  measure  of  safety  program  effectiveness  is  the  response  from  the  entire  organization  to 
questions  about  the  quality  of  the  management  systems  that  have  an  effect  on  human  behavior 
relating  to  safety. 

3.  Training  and  Competency 

The  workforce’s  perception  of  the  general  level  of  workers’  qualifications,  skills,  and 
knowledge  is  the  essence  of  this  theme.  Training  is  a  key  component,  both  directly  through  its 
effect  on  workers’  competency  to  perform  their  work  functions,  but  also  indirectly  through 
influencing  perceptions  about  management’s  commitment  to  safe  and  reliable  work  systems.  In 
fact,  Cooper  and  Phillips  (2004)  demonstrated  that  workers’  perceptions  of  the  significance  of 
safety  training  were  the  most  important  safety  climate  factor  predicting  actual  safety  behavior. 

In  recent  years,  many  high-reliability  organizations  have  embarked  upon  a  process  of 
multiskilling  the  workforce.  This  process  can  be  seen  positively  or  negatively  by  the  workforce, 
depending  upon  whether  it  is  properly  applied  and  resourced.  Concurrent  with  this  development, 
HROs  have  also  emphasized  competence  in  nontechnical  skills  (e.g.,  leadership  and  decision 
making)  that  are  regarded  as  contributing  factors  to  safe  operations  (Helmreich  &  Merritt,  1998). 
These  are  usually  taught  in  crew  resource  management  training  programs  (Flin,  O’Connor,  & 
Crichton,  2008)  and,  as  such  training  becomes  more  widespread,  this  aspect  of  the  skill  base  also 
may  need  to  be  incorporated  into  a  competence  factor. 

4.  Risk 

The  risk  literature  focuses  on  three  aspects  of  risk: 

•  risk  perception  -  the  extent  to  which  a  person  believes  a  particular  activity  or 
situation  is  risky; 

•  risk  tolerance  -  the  extent  to  which  a  person  is  willing  to  engage  in  risky 
behaviors;  and 

•  risk  behavior  -  the  frequency  with  which  a  person  engages  in  risky  activities  or 
situations  (Kivmaki,  Kalimo,  &  Sahninen,  1995;  Dedobbeleer  &  Beland,  1991). 

In  the  aviation  industry,  pilots  who  reported  higher  risk  perception  also  reported  lower 
risk  tolerance  towards  experiencing  risky  aviation  activities,  particularly  for  weather-related 

situations  (Hunter,  2002).  Importantly,  the  number  of  experienced  hazardous  events  was 
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consistent  with  self-reported  risk  tolerance  (e.g.,  those  pilots  who  rated  various  aviation  activities 
as  more  risky  than  the  average  pilot  experienced  fewer  hazardous  events  [Hunter,  2002]). 
Additionally,  risk  tolerance  was  a  significant  factor  in  a  model  attempting  to  distinguish  between 
pilots  who  had  experienced  an  accident  and  those  who  had  not  (Platenius  &  Wilde,  1989).  Pilots 
involved  in  an  accident  were  more  likely  to  endorse  statements  such  as  “I  fly  in  spite  of  advice 
from  others,”  “if  a  pilot  doesn’t  take  occasional  risks,  they  won’t  leam  to  get  out  of 
emergencies,”  and  “physically  capable  people  can  take  more  risks.” 

Studies  of  risk  perception  in  other  industries  have  shown  that  workers  have  fairly 
accurate  perceptions  of  the  risks  they  face  (Flin  et  ah,  1996).  Moreover,  just  as  in  the  aviation 
environment,  higher  threat  perception  is  positively  related  to  safe  behaviors.  For  example, 
Goldberg,  Dar-El,  and  Rubin  (1991)  found  that  a  high  threat  perception  was  related  to  readiness 
to  participate  in  safety  programs,  the  relationship  was  mediated  by  coworker  support  for  safety. 
Workers  who  sensed  high  coworker  support  for  safety  were  more  likely  to  be  positively  oriented 
towards  greater  participation.  Cheyne,  Cox,  Oliver,  and  Tomas  (1998)  found  that  perception  of 
workplace  hazards  did  not  have  a  direct  effect  on  levels  of  safety  activity  by  the  workforce. 
However,  an  indirect  effect  was  found,  such  that  higher  appraisals  of  workplace  hazards  were 
related  to  more  positive  perceptions  of  individual  responsibility,  which,  in  turn,  affects  levels  of 
safety  activity.  Similarly,  Dedobbeleer  and  Beland  (1991)  speculate  that  workers’  perceptions  of 
risk  and  control  may  be  highly  related  to  workers’  involvement  and  responsibility  for  safety. 
Overall  then,  it  seems  that  risk  perceptions  and  safety  behaviors  are  positively  related  to 
perceptions  of  involvement. 

In  a  study  of  U.S.  coal  mines,  the  opposite  effect  also  appears  to  hold  true.  Brown,  Willis 
and  Prussia  (2000)  found  that  safety  hazards  directly  caused  accidents  and  indirectly  influenced 
employees’  perceptions  of  the  safety  climate.  Higher  perceptions  of  hazard  are  associated  with 
reduced  perceptions  of  manager  and  supervisor  attitudes  towards  safety.  Higher  perceptions  of 
risk  are  also  associated  with  increased  production  pressure,  which  leads  to  unsafe  behaviors. 

5.  Work  Pressure 

Factors  relating  to  work  pace,  workload  and  work  pressure,  and  production  pressure  are 
common  in  safety  climate  surveys,  and  for  good  reason.  This  factor  has  consistently  shown  a 

positive  relationship  with  accident  rates.  For  example,  Cooper  and  Phillips  (2004)  found 
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significant  differences  in  perceptions  of  work  pace  between  accident-involved  and  non-accident- 
involved  workers.  Diaz  and  Cabrera  (1997)  found  that  employees’  perceptions  of  the 
organization’s  philosophy  of  either  production  or  safety,  was  the  second  most  important  factor 
(after  organizational  policies  towards  safety)  in  predicting  safety  performance.  Pfeifer,  Stefanski, 
and  Grerther  (1976)  questionnaire  results  indicate  that  supervisors  in  low-accident  rate  mines 
were  significantly  less  inclined  to  push  hard  for  production  or  to  cut  corners  on  safety.  Sanders, 
Patterson,  and  Peay  (1976)  found  that  increased  levels  of  production  pressure  were  associated 
with  increased  lost  time  injury  rates. 

In  his  sociological  investigations  into  the  causes  of  accidents  in  the  U.K.’s  offshore  oil 
industry,  Wright  (1986)  found  that  perceptions  of  performance  pressure  can  lead  workers  to 
believe  that  engaging  in  short-cut  behavior  is  an  expected  or  required  part  of  the  job.  Workers 
who  perceive  a  high  degree  of  perfonnance  pressure  will  focus  their  attention  on  completing  the 
work  and  focus  less  on  the  safety  of  their  work  procedures. 

6.  Procedures/Rules 

Guldenmund  (2000)  identified  procedural  and  rule  compliance  as  one  of  the  most 
frequently  occurring  themes  in  his  review  of  safety  climate  research.  Perceptions  of  safety  rules, 
attitudes  to  rules  and  compliance,  and  violation  of  procedures  are  addressed.  This  theme  is  also 
related  to  risk-taking  behaviors,  as  these  can  involve  rule  breaking.  Some  of  these  factors  have 
been  shown  to  relate  to  accident  involvement  in  safety  climate  surveys  (e.g.,  Lee,  1998),  but 
causal  relationships  remain  more  obscure  and  are  likely  to  be  influenced  by  supervisory  behavior 
and  work  pressure  variables.  This  issue  has  received  scrutiny  in  studies  of  worksite  safety 
(Reason,  1998),  suggesting  that  procedural  compliance  is  an  issue  that  should  be  addressed  in 
measures  of  safety  climate. 

Like  many  of  the  other  safety  climate  themes,  procedural  and  rule  compliance  appears  to 
be  strongly  influenced  by  perceptions  of  management  and  labor-management  relations.  For 
example,  Thompson,  Hilton,  and  Witt  (1998)  found  that  the  supervisor’s  role  in  promoting 
workplace  safety  is  achieved  by  affecting  the  perceived  level  of  fairness  in  their  organization’s 
climate,  which,  in  turn,  impacts  on  workforce  compliance  with  safety  rules.  Likewise,  Simard 
and  Marchand  (1997)  found  that  a  cohesive  and  cooperative  workgroup  relationship  is  by  far  the 

most  important  variable  in  terms  of  predicting  workgroups’  propensity  to  comply  with  safety 
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rules.  Cooperative  relationships  are  characterized  by  more  open  communications  and  positive 
team  spirit. 

Overall,  the  studies  reviewed  reveal  a  number  of  interesting  findings  relating  to  the  nature 
of  workforce  motivation  and  the  changing  roles  of  managers,  supervisors,  and  workers.  It  seems 
that  employees  are  now  expected  to  do  more  than  just  comply  with  rules  and  regulations.  They 
are  expected  to  act  proactively,  be  personally  committed  to  safety,  take  responsibility  and 
ownership  for  safety,  and  be  committed  to  corporate  safety  goals. 

To  summarize,  the  six  themes  reviewed  above  represent  those  that  are  most  commonly 
included  in  measures  of  safety  climate.  It  also  can  be  seen  that  the  literature  supports  the  notion 
that  these  themes  should  be  considered  when  assessing  the  safety  climate  of  an  organization. 
Thus,  a  measure  of  safety  climate  should  include  items  that  address  each  of  these  themes.  In  the 
next  section,  the  evidence  linking  safety  climate  to  other  measures  of  safety  performance  will 
be  discussed. 

D.  CORRELATING  SAFETY  CLIMATE  WITH  OTHER  MEASURES  OF  SAFETY 

PERFORMANCE 

In  addition  to  establishing  the  construct  validity  of  a  safety  climate  questionnaire,  it  is 
also  necessary  to  detennine  the  discriminate  validity.  If  the  tool  is  insufficiently  sensitive  in  its 
ability  to  differentiate  between  organizations  or  personnel  with  different  levels  of  safety 
performance,  then  the  tool  is  of  limited  usefulness.  The  discriminate  validity  can  be  assessed  by 
correlating  the  data  from  the  questionnaire  with  a  criterion  variable  such  as  accidents,  or  other 
safety-related  behavior  (Guldenmund,  2007). 

In  recent  years,  a  large  number  of  research  studies  have  been  conducted  that  have  sought 
to  examine  the  contribution  of  safety  climate  to  accidents.  The  challenge  facing  researchers  has 
been  to  highlight  measurable  dimensions  of  safety  climate  that  can  be  used  to  identify,  in 
advance,  the  strengths  and  weaknesses  within  an  organization  that  influence  the  likelihood  of 
accidents  occurring.  A  variety  of  different  criteria  are  used  upon  which  to  base  evaluations  of 
organizational  effectiveness  in  preventing  accidents  including:  company  accident  statistics; 
comparison  of  high  and  low  accident  rate  plants,  and  evaluation  of  plants  with  outstanding  safety 
records;  and  self-reported  safety  behaviors  and  safety  attitudes. 
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1.  Company  Accident  Statistics 

Over  the  past  two  decades,  researchers  have  demonstrated  relationships  between  a  variety 
of  safety  climate  factors  and  accident  rates  across  a  range  of  high-risk  industries.  Such  studies 
have  shown  that  the  degree  of  safety  program  development  and  workers’  safety  initiative  were 
related  to  lower  work  accident  and  injury  rates  (Simard  &  Marc  hand,  1994;  Zohar,  2000;  Dwyer 
&  Raftery,  1991;  Donald  &  Canter,  1994;  Mearns,  Rundmo,  Flin,  Gordon,  &  Fleming,  2004). 

Johnson  (2007)  provides  support  for  safety  climate  as  a  viable  construct  and  as  a 
predictive  indicator  of  safety-related  outcomes.  This  study  used  the  responses  of  292  employees 
at  three  locations  of  a  heavy  manufacturing  organization  to  complete  the  16-item  Zohar  Safety 
Climate  Questionnaire  (ZSCQ;  Zohar  &  Luria,  2005).  In  addition,  safety  behavior  and  accident 
experience  data  were  collected  for  five  months  following  the  survey  and  were  analyzed  to 
identify  correlations,  associations,  internal  consistency,  and  factorial  structures.  Results  revealed 
that  safety  climate,  as  measured  by  the  ZSCQ,  served  as  an  effective  predictor  of  safety-related 
outcomes  (behavior  and  accident  experience). 

There  are  problems  with  using  accident  rates  because  they  are  a  notoriously  unreliable 
measure  of  an  organization’s  true  safety  performance.  Used  alone,  they  can  be  a  misleading 
indicator  of  the  effectiveness  of  a  safety  program  (Thompson  et  ah,  1998).  Four  main  difficulties 
have  been  identified: 

1.  restriction  of  variance:  accidents  are  rare  occurrences,  which  can  make  the  data 
unreliable; 

2.  random  events  that  are  not  under  the  direct  control  of  personnel  can  sometimes 
intercede  to  cause  accidents,  which  also  produces  unreliability; 

3.  accidents  may  not  be  consistently  recorded  across  organizations,  and  over  and 
under  recording  causes  unreliability;  and 

4.  the  measurement  of  accident  severity  is  often  a  highly  subjective  issue  and, 
therefore,  also  causes  unreliability  (Thompson  et  ah,  1998;  Witt,  Heilman,  & 
Hilton,  1994;  Zohar,  1980).  For  these  reasons,  many  researchers  have  abandoned 
the  use  of  accident  frequency  data  and  accident  severity  rates  as  a  measure  of 
safety  effectiveness,  in  favor  of  measures  such  as  self-reported  accident 
involvement  (see  later  for  a  discussion). 
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2.  Comparison  of  High  and  Low  Accident  Rate  Plants 

Other  studies  have  compared  high-  and  low-accident-rate  plants  (or  evaluated  plants  with 
outstanding  safety  records)  as  their  criteria  upon  which  to  base  judgments  of  effectiveness. 
However,  it  is  difficult  to  identify  those  variables  that  are  crucial  to  their  outstanding 
performance,  as  opposed  to  those  that  are  simply  associated  with  it.  The  differences  highlighted 
between  good  and  bad  companies  may  only  be  a  fraction  of  the  total  and  these  differences  may 
change  over  time  (Meams,  Flin,  &  O’Connor,  2001).  Furthermore,  the  causal  relationship 
between  variables  and  outcomes  are  not  proven,  it  is  often  difficult  to  say  which  is  the 
independent  variable  and  which  the  dependent  variable  (Hale  &  Hovden,  1998).  Nonetheless,  the 
early  studies  that  compared  high-  and  low-accident-rate  organizations  did  identify  issues  that 
have  been  supported  by  the  other  more  objective  approaches  to  validation. 

As  with  much  of  the  other  literature  on  safety  culture  and  climate,  in  studies  that  compare 
high-  and  low-accident-rate  plants,  management  emerges  as  a  key  and  consistent  determinant  of 
safety  performance.  In  particular,  management’s  commitment  to  safety  was  found  to  be  greater 
in  low-accident-rate  plants  than  in  the  high-accident-rate  plants  (Cohen,  Smith,  &  Cohen,  1975; 
Smith  et  ah,  1978;  Cohen  &  Cleveland,  1983).  Commitment  is  expressed  through  the  allocation 
of  resources  to  safety  and  health,  and  more  active  involvement  and  participation  by  management 
in  safety  program  matters  (Cohen  &  Cleveland,  1983;  Diaz  &  Cabrera,  1997).  Aside  from  their 
commitment,  a  manager’s  style  of  leadership  (centralized  or  decentralized),  a  manager’s 
involvement  with  workers,  and  labor-management  relations  appear  to  be  the  key  factors  that 
distinguish  the  high  from  the  low  performers  (Braithwait,  1985;  DeMichiei,  Langton,  Bullock,  & 
Wiles,  1982). 

High  levels  of  management  control  over  work  organization  and  task  structure,  serves  to 
reduce  worker  autonomy  and  worker  integration,  which  is  associated  with  higher  accident  rates 
(Dwyer  &  Raftery,  1991;  Braithwait,  1985;  Sanders  et  ah,  1976).  Worker  involvement  in 
decision  making  and  policy  setting  serves  to  reverse  this  trend.  Cohen  and  Cleveland  (1983),  in 
their  analysis  of  the  top-performing  companies,  found  that  in  the  best  companies  in  the  USA, 
workers  were  more  involved  in  the  decision-making  process.  They  had  a  direct  channel  to 
communicate  their  thoughts  and  ideas  to  management;  management  was  more  receptive  and 
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responsive  to  requests  and  suggestions;  and  more  frequently  solicited  input  concerning  policies 
and  procedures  from  workers. 

Conversely,  poor  labor  management  relations  are  associated  with  significantly  higher 
self-reported  violation  rates  and  higher  injury  rates  (Gaertner  et  ah,  1987).  Direct  channels  of 
communication  between  labor  and  management  appear  to  be  an  important  factor.  Smith  et  al. 
(1978)  found  that  management  of  low-accident-rate  plants  seemed  to  have  a  greater  level  of  one- 
to-one  interaction  with  their  employees,  while  in  high-accident-rate  plants,  management  more 
often  relied  upon  committees  to  interact  with  employees.  Similar  findings  are  reported  by  Cohen 
and  Cleveland  (1983),  who  found  that  the  top-performing  companies  provided  direct  and 
immediate  channels  of  communication  and  positive  employee-management  interactions. 

3.  Self-Reported  Safety  Behavior 

Self-reported  safety  behavior  and  safety  attitudes  are  an  alternative  to  relying  on  mishaps 
data  to  evaluate  the  effectiveness  of  an  organization’s  safety  program.  Thompson  et  al.  (1998) 
suggested  that  minor  workplace  accidents  often  go  unreported,  yet  these  events  may  be  the  best 
indicators  of  improving  (or  worsening)  safety  conditions  that  might  eventually  lead  to  serious 
injury.  Members  of  the  workforce  are  likely  to  be  sensitive  to  the  type  and  frequency  of 
accidents  that  go  under-reported  and,  as  such,  their  sense  of  workplace  safety  conditions  might 
be  a  better  indicator  of  safety  risk  than  the  routine  accident  reports  (Thompson  et  al.,  1998). 
However,  this  approach  is  complicated  by  questions  about  who  is  in  the  best  position  to  provide 
the  ratings.  Some  studies  use  supervisory  or  managerial  attitudes  towards  employees’  safety 
behavior  (Simard  &  Marchand,  1994;  1995;  1997),  others  use  employees’  ratings  of  their  own 
attitudes  and  those  of  their  managers  and  supervisors.  (Andriessen,  1978;  Thompson  et  al., 
1998).  However,  the  judgments  are  likely  to  be  influenced  by  attribution  bias  (i.e.,  attribute  poor 
performance  to  other  people  rather  than  one’s  self).  Employees  may  have  little  direct  evidence  of 
management  attitudes  to  safety;  rather,  they  may  infer  managerial  attitudes  on  the  basis  of  their 
experiences  of  supervision.  Similarly,  management  who  are  removed  from  operations  may  have 
little  direct  experience  of  employee  opinions  (Clarke,  1999).  In  most  of  the  studies  published  in 
the  literature,  no  explanation  is  given  for  the  choice  of  respondent  used.  It  seems  that 
methodological  convenience,  as  much  as  theoretical  reasoning,  is  driving  the  decision. 
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Climate  is  linked  to  safety  outcomes  through  behavior-outcome  expectancies.  These 
expectancies  guide  behavior  through  learning.  Evidence  has  shown  that  safety  behaviors  are 
positively  related  to  company  safety  records,  and  reduced  accident  and  injury  rates.  Cooper  and 
Phillips  (2004)  established  an  empirical  link  between  a  limited  set  of  safety  climate  perceptions 
and  actual  safety  behavior.  They  also  demonstrated  that  changes  in  climate  perceptions  do  not 
necessarily  reflect  changes  in  levels  of  behavioral  safety  performance.  Equally,  changes  in  safety 
behavior  are  not  necessarily  reflected  in  safety  climate  perceptions.  Such  results  remind  us  that  it 
is  reductions  in  the  frequency  of  unsafe  behaviors  and  their  antecedents  (i.e.,  unsafe  conditions 
or  situations)  that  reduce  the  opportunity  for  accidents  to  occur,  not  perceptions  about  how  safety 
is  operationalized.  The  findings  strongly  suggests  that  industry  should  focus  its  primary  safety 
improvement  effort  on  changing  unsafe  situations  and  conditions,  as  well  as  people’s  safety 
behavior  at  all  organizational  levels,  rather  than  concentrating  on  improving  people’s  attitudes, 
beliefs,  and  perceptions  about  safety.  The  authors  argue  for  the  use  of  a  variety  of  safety 
performance  outcome  variables,  rather  than  relying  primarily  on  self-report  instruments. 

More  recently,  a  number  of  meta-analytic  studies  have  been  conducted  in  order  to 
evaluate  the  relationship  between  safety  climate  as  a  holistic  construct  and  accident  rates.  Clarke 
(2006)  used  a  meta-analysis  to  examine  the  validity  of  the  relationship  between  safety  climate 
and  safety  performance  and  occupational  accidents  and  injuries  (r=0.2).  The  study  found  support 
for  the  link  between  organizational  safety  climate  and  employee  safety  participation;  however, 
the  links  to  accident  involvement  were  found  to  be  weak.  Not  surprisingly,  the  relationship 
between  safety  climate  and  accident  involvement  was  moderated  by  the  study  design,  such  that 
only  prospective  design,  (i.e.,  studies  in  which  accidents  were  measured  following  the 
measurement  of  safety  climate)  demonstrated  validity  generalization.  Those  studies  that  were 
retrospective  (i.e.,  studies  where  the  measurement  of  accidents  or  injuries  is  taken  before  the 
measurement  of  safety  climate)  did  not  show  such  a  link.  This  finding  holds  with  the  principle 
that  climate  predicts  accidents  and  not  the  other  way  around. 

Similarly,  Nahrgang,  Morgeson,  and  Hofmann  (2007)  conducted  a  meta-analysis  that 
examined  the  relationship  between  safety  perfonnance  and  its  antecedents — including  safety 
climate — in  order  to  establish  which  factors  are  more  influential  in  establishing  strong  safety 
performance  (accident  and  injury  rates,  and  positive  safety  behavior).  Results  indicate  safety- 
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related  antecedents  (such  as  risks  and  hazards,  safety  prevention,  and  safety  involvement)  and 
general  antecedents  (leadership  and  commitment)  have  moderate  to  strong  relationships  with 
safety  climate.  Leadership  and  safety  climate  both  demonstrate  moderately  negative  relationships 
to  accidents  and  injuries,  and  moderately  positive  relationships  with  positive  safety  behavior. 
However,  risks  and  hazards  is  the  only  safety-related  antecedent,  which  correlates  with  accidents 
and  injuries.  Leadership  also  demonstrates  moderate  relationships  with  the  two  safety  outcomes 
of  accidents  and  injuries  and  positive  safety  behavior.  Overall,  safety  climate  negatively 
correlates  with  accidents  and  injuries. 

The  studies  suggest  that  the  behaviors  that  underlie  safety  culture  are  even  more  critical 
as  indicators  of  an  organization’s  safety  perfonnance  than  other,  more  traditional,  measures  such 
as  accident  or  incident  rates.  Herein  lies  the  greatest  strength  of  the  concept.  Safety  climate 
introduces  the  notion  that  the  likelihood  of  accidents  occurring  can  be  predicted  on  the  basis  of 
certain  organizational  factors.  These  organizational  factors  can  be  used  as  leading  indicators  to 
identify,  in  advance,  the  strengths  and  weaknesses  within  an  organization  that  influence  the 
likelihood  of  accidents  occurring.  Once  weaknesses  are  identified,  remedial  actions  can  be  taken 
(Flin  et  al.,  2000). 

To  summarize,  as  would  be  hoped,  there  would  appear  to  be  a  link  between  safety 
climate  and  mishaps.  However,  the  relationship  is  not  as  strong  as  may  be  expected,  part  of 
which  can  be  attributed  to  the  difficulties  in  obtaining  accurate  and  truthful  mishap  data.  In  the 
next  section,  the  safety  climate  research  that  has  been  carried  out  in  aviation  will  be  delineated, 
with  particular  emphasis  on  the  tools  used  by  U.S.  Naval  aviation. 

E.  SAFETY  CLIMATE  ASSESSMENT  IN  AVIATION 

Wiegmann  and  colleagues  (Wiegmann,  Zhang,  von  Thaden,  Sharma,  &  Gibbons,  2004) 
report  that  “few  formally  documented  efforts  have  been  made  to  assess  safety  culture  within  the 
aviation  industry,  with  the  notable  exception  of  military  aviation”  (p.  117).  This  finding  is 
surprising,  given  that  the  civilian  aviation  industry  has  led  other  high  reliability  in  developing 
and  utilizing  a  number  of  human-focused  safety  programs  (e.g.,  crew  resource  management). 
However,  the  extent  to  which  safety  climate  surveys  are  being  used  in  commercial  aviation  is 
difficult  to  assess.  It  is  recognized  that  there  are  undoubtedly  many  aviation  consultancy 

companies  carrying  out  safety  climate  assessments.  However,  because  this  information  is  not  in 
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the  public  domain,  this  work  cannot  be  reviewed.  Below  is  a  discussion  of  10  studies  carried  out 
in  commercial  aviation  that  report  a  safety  climate  evaluation,  and  are  available  in  the 
public  domain. 

1.  Commercial  Pilots 

The  Australian  Transportation  Safety  Board  (2004)  and  Evans,  Glendon,  and  Creed 
(2007)  both  reported  on  a  study  examining  the  safety  climate  of  Australian  pilots.  The 
questionnaire  consisted  of  six  safety  factors  (management  commitment,  training,  equipment  and 
maintenance,  rules  and  procedures,  communication,  and  schedules),  each  with  five  items.  These 
factors  were  based  upon  previous  safety  climate  research  and  input  from  aviation  safety  experts. 
Data  from  half  of  the  sample  were  used  in  an  exploratory  factor  analysis  that  resulted  in  a  three 
factor  model  of:  management  commitment  and  communication,  safety  training  and  equipment, 
and  maintenance.  A  confirmatory  factor  analysis  on  the  remaining  half  of  the  sample  showed  the 
three  factor  model  to  be  an  adequate  fit  to  the  data.  Finally,  the  responses  from  different  types  of 
pilots  (regular  public  transport,  charter,  or  aerial  work  such  as  emergency  medical  services  or 
agriculture)  were  compared  on  each  of  the  four  identified  safety  climate  factors.  No  significant 
differences  between  the  groups  were  found.  The  Australian  Transportation  Safety  Board  (2004) 
concluded  that  this  was  due  to  a  single  professional  safety  climate  for  pilots  as  a  group, 
regardless  of  the  organization  for  whom  they  worked. 

Gibbons,  von  Thaden,  and  Wiegmann  (2006)  developed  a  questionnaire  designed  to 
assess  safety  culture  within  the  context  of  airline  flight  operations.  The  survey  consisted  of  84 
items,  grouped  into  five  themes  (organizational  commitment,  management  involvement, 
employee  empowerment,  reward  systems,  and  reporting  systems).  The  survey  was  designed  by 
examining  the  content  of  safety  climate  questionnaires  that  have  been  used  in  other  HROs.  A 
total  of  503  responses  were  received  from  a  single  company.  After  discarding  29  items  and  using 
a  confirmatory  factor  analysis  technique,  the  analysis  eventually  resulted  in  structure  of  four 
broad  factors  (organizational  commitment,  operations  personnel,  informal  safety  system,  and 
formal  safety  system),  with  three  subfactors  in  each.  The  authors  attribute  the  difficulty  in 
establishing  a  stable  factor  structure  with  the  analysis  to  issues  in  item  writing  (e.g.,  ambiguity, 
items  that  did  not  relate  well  to  the  target  population).  Another  issue  not  mentioned  in  the  paper 
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is  the  relatively  low  ratio  of  items  to  responses.  No  analysis  of  the  revised  questionnaire 
was  reported. 


2.  Cabin  Staff 

Kao,  Stewart,  and  Lee  (2009)  developed  a  23-item  questionnaire  to  assess  the  safety 
climate  attitudes  of  Taiwanese  cabin  crews.  The  questionnaire  was  designed  to  assess  the 
following  safety  climate  themes:  management  commitment  towards  safety,  cabin  work 
environment,  rule  compliance,  crewmember  involvement  and  participation,  accident 
investigation,  and  injury  incidence.  The  items  were  based  upon  previous  safety  climate  research. 
A  total  of  331  responses  were  obtained  from  cabin  crews  from  four  major  Taiwanese  airlines. 
Using  a  structural  equation  modeling  approach,  the  researchers  found  an  acceptable  level  of  fit 
with  the  proposed  factors.  High  management  commitment  to  safety  was  significantly  related  to 
high  crewmember  participation  in  safety,  and  that  safe  cabin  work  environment  was  significantly 
related  to  crewmember’s  individual  behavior.  However,  the  findings  did  not  reveal  a  direct 
relationship  between  management  commitment  and  injury  incidence. 

3.  Ground  Handlers 

Diaz  and  Cabrera  (1997)  developed  a  40-item  safety  climate  questionnaire  for  aviation 
ground  handlers,  based  upon  the  work  of  Zohar  (1980).  Following  a  PCA  on  the  data  collected 
from  166  ground  handling  personnel  at  a  Spanish  airport,  six  factors  were  identified:  company 
policy  towards  safety,  emphasis  on  productivity  versus  safety,  group  attitudes  to  safety,  specific 
strategies  of  prevention,  safety  level  perceived  in  the  airport,  and  safety  level  perceived  on  the 
job.  It  was  found  that  the  questionnaire  was  able  to  discriminate  between  organizations  with 
different  levels  of  safety. 

Ek  and  Akselsson  (2007)  evaluated  the  safety  culture  in  the  ramp  division  of  a  ground 
handling  company.  A  109-item  questionnaire  was  developed  that  addressed  nine  aspects  of 
safety  climate:  working  situation,  communication,  learning,  reporting,  justness,  flexibility, 
attitudes  towards  safety,  safety-related  behaviors,  and  risk  perception.  Data  were  collected  from 
50  men  employed  by  a  single  ground  handling  company.  Acceptable  levels  of  internal 
consistency  were  found  for  each  factor.  They  concluded  that  the  safety  climate  was  good,  but 
poorer  than  desired  by  managers. 
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4. 


Aviation  Maintainers 


As  part  of  a  larger  research  project,  McDonald,  Corrigan,  Daly,  and  Cromie  (2000) 
designed  and  utilized  a  safety  climate  questionnaire  to  survey  aviation  maintainers.  The 
questionnaire  was  adapted  from  the  one  developed  by  Diaz  and  Cabrera  (1997;  described  above). 
A  36-item  questionnaire  was  designed  based  upon  a  factor  analysis  of  69  items  (this  analysis  was 
not  reported).  A  total  of  622  responses  were  obtained  from  aviation  maintainers  from  four 
companies.  Significant  differences  in  climate  were  found  between  different  occupational  groups. 
McDonald  et  al.  (2000)  reported  that  the  data  provided  evidence  of  a  strong  professional 
subculture,  which  spanned  all  of  the  four  companies  that  participated  in  the  study.  Further,  this 
subculture  is  relatively  independent  of  the  organization.  It  was  postulated  that  the  subculture  is 
likely  to  mediate  between  the  organization’s  safety  management  system  and  safety  outcome. 

5.  Air  Traffic  Controllers 

Gordon,  Kirwan,  Mearns,  Kennedy  and  Jensen  (2007)  describe  a  pilot  study  of  a  climate 
survey  designed  for  use  by  European  air  traffic  controllers  (ATC).  The  questionnaire  consisted  of 
59  items  of  13  elements  designed  around  three  themes  (priority  of  safety,  involvement  of  safety, 
and  learning  from  safety).  The  items  were  selected  based  upon  a  literature  review,  50  interviews 
with  ATC  personnel,  and  input  from  subject  matter  experts  on  the  final  items  to  be  included.  The 
questionnaire  was  piloted  with  119  responses  obtained.  Following  an  exploratory  factor  analysis, 
an  8-factor  questionnaire  resulted  (see  Table  1  for  a  description  of  the  factors).  Gordon  et  al. 
(2007)  acknowledge  that  the  sample  was  small,  and  they  state  that  a  larger  validation  study  will 
be  carried  out. 


6.  Combined  Commercial  Aviation 

Patankar  (2003)  evaluated  the  safety  climate  of  a  stratified  sample  of  399  personnel 

(flight  operations,  maintenance,  and  other  personnel)  from  a  single  aviation  company  using  a 

common  safety  climate  questionnaire.  The  questionnaire  was  based  upon  the  cockpit 

management  attitude  questionnaire,  (CMAQ;  Gregorich,  Helmreich,  &  Wilhelm,  1990),  the 

maintenance  resource  management/technical  operations  questionnaire  (MRM/TOQ;  Taylor, 

2000),  and  the  CSAS  (discussed  in  detail  below).  After  a  factor  analysis  (no  details  of  this  were 

reported),  eight  factors  emerged:  pride  in  company,  professionalism,  safety  opinions,  supervisor 
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trust  and  safety,  effects  of  my  stress,  need  to  speak  up,  safety  compliance,  and  hazard 
communication.  Significant  differences  were  found  between  flight  operations,  maintenance,  and 
“other”  personnel  with  regard  to  the  factors  of  pride  in  company,  safety  opinions,  and  supervisor 
trust.  Patankar  (2003)  concluded  that,  overall,  the  respondents  were  proud  to  work  for  the 
company,  trusted  management,  and  believed  that  safety  is  a  result  of  collective  efforts.  It  also 
was  commented  on  that  both  flight  and  maintenance  personnel  had  a  high  sense  of  personal 
responsibility  for  flight  safety. 

In  a  later  study,  the  data  collected  by  Patankar  (2003;  called  company  A)  was  compared 
to  237  responses  collected  at  another  company  (called  company  B;  Kelly  &  Patankar,  2004).  It 
was  found  that,  overall,  there  was  a  more  positive  safety  climate  at  company  A  than  company  B. 
However,  this  finding  was  partially  attributed  to  company  A  having  older  and  more  experienced 
pilots  and  mechanics  than  company  B. 

Block,  Sabin,  and  Patanakar  (2007)  reanalyzed  the  responses  obtained  from  the  281  pilots 
from  the  Patanakar  (2003)  sample.  The  purpose  was  to  examine  whether  the  data  supported  what 
Block  et  al.  (2007)  described  as  the  purpose-alignment-control  (PAC)  model.  A  pair  of  experts 
recoded  the  Patankar  (2003)  survey  items  in  accordance  with  the  PAC  model.  The  proposed 
factors  were  tested  using  a  structural  equation  modeling  methodology.  The  main  drivers  of  safety 
outcomes  were  organizational  affiliation  (similar  to  pride  in  company  from  Patankar,  2003)  and 
proactive  management  (partially  derived  from  safety  opinion  factor  from  Patankar,  2003). 
Organizational  affiliation  was  directly  influenced  by  communication,  and  proactive  management 
was  influenced  by  training  effectiveness  and  relational  supervision. 

Gill  and  Shergill  (2004)  conducted  a  safety  climate  review  across  the  New  Zealand 
commercial  aviation  industry.  The  safety  climate  questionnaire  they  developed  included 
questions  designed  to  address  two  themes:  organizations’  approach  to  safety  management 
(26  items)  and  “safety  management  systems,  and  safety  culture  in  organizations”  (26  items).  A 
factor  analysis  of  464  responses  was  run  independently  on  each  theme.  The  “safety  management 
systems”  theme  was  found  to  consist  of  four  factors:  positive  safety  practices;  safety  education; 
implementation  of  safety  policies  and  procedures;  and  individual’s  safety  responsibilities.  The 
“safety  culture  in  organizations”  theme  was  also  found  to  consist  of  four  subfactors: 
organizational  dynamics  and  positive  safety  practices;  regulator’s  role;  luck  and  safety;  and 


20 


safety  management,  training,  and  decision  making.  The  main  findings  from  the  study  were  that 
pilots  believed  luck  and  safety  to  be  the  most  important  factor  in  aviation  safety,  and  employers 
were  not  perceived  to  be  placing  much  importance  on  safety  management  systems  and 
safety  culture. 

As  can  be  seen  from  the  review  of  the  safety  climate  literature  described  earlier,  a 
summarization  of  the  research  carried  out  in  commercial  aviation  indicates  that  the  themes  are 
not  dissimilar  from  those  identified  in  nonaviation  HRO  safety  climate  research.  The  commercial 
aviation  studies  reviewed  generally  describe  the  development  of  “new”  research  questionnaires 
that,  in  most  cases,  have  only  been  used  once  with  a  maximum  of  a  few  hundred  respondents, 
and  represent  a  one-time  safety  climate  assessment.  Furthermore,  no  attempts  were  made  to 
examine  the  discriminate  validity  of  the  measures  by  correlating  the  survey  data  with  other  safety 
performance  measures  (e.g.,  accident  rate).  In  contrast,  U.S  Naval  aviation  has  been  collecting 
data  on  safety  climate  continuously  since  2000.  The  tools  used  to  assess  safety  climate  in  Naval 
aviation  will  be  discussed  in  the  next  section. 

F.  SAFETY  CULTURE  ASSESSMENT  IN  NAVAL  AVIATION 

The  U.S.  Navy  utilizes  two  different  tools  to  assess  safety  climate  in  aviation.  The  CSAS 
is  used  to  obtain  feedback  from  aviators,  and  the  Maintenance  Climate  Assessment  Survey 
(MCAS)  to  obtain  information  from  aviation  maintainers.  It  should  also  be  mentioned  that, 
although  not  discussed  in  detail  here,  the  Navy  also  conducts  safety  climate  workshops  with 
aviation  squadrons.  The  facilitators  (specially  trained  senior  naval  aviators)  conduct 
observations,  interviews,  and  focus  groups  with  squadron  personnel.  The  purpose  is  to  identify 
potential  hazards  that  may  interfere  with  mission  accomplishment  (see  O’Connor  &  O’Dea, 
2007,  for  more  details).  However,  this  program  is  run  independently  of  the  safety  climate  survey. 

The  safety  culture  questionnaires  were  developed  by  researchers  at  the  Naval 
Postgraduate  School  in  Monterey,  California  (Desai,  Roberts,  &  Ciavarelli,  2006).  Both 
questionnaires  are  completed  online,  and  responses  are  obtained  for  each  item  on  a  5-point  Likert 
scale  from  1  (strongly  disagree)  to  5  (strongly  agree).  In  2004,  Vice  Admiral  Zortman  declared 
the  MCAS  and  CSAS  mandatory  for  all  squadrons  to  complete  semiannually  and  within  30  days 
following  a  change  of  command  (Zortman,  2004).  The  results  of  a  squadron’s  survey  are  only 
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available  to  the  Commanding  Officer  (CO).  However,  aggregated  data  is  made  available  to  all 
COs  for  comparison  of  their  squadron’s  perfonnance  with  their  peers. 

The  theoretical  background  underpinning  the  questionnaires  is  based  upon  the  work 
carried  out  by  Roberts  et  al.  on  HROs  (Desai  et  al.,  2006).  Libuser  (1994)  developed  a  theoretical 
Model  of  Organizational  Safety  Effectiveness  (MOSE)  that  identified  five  major  areas  relevant 
to  organizations  in  managing  risk  and  developing  a  climate  to  reduce  accidents.  The  five  MOSE 
areas  are: 

•  Process  Auditing  -  a  system  of  ongoing  checks  to  monitor  hazardous  conditions 
(e.g.,  “My  command  conducts  adequate  reviews  and  updates  of  safety  standards 
and  operating  procedures.”). 

•  Reward  System  -  expected  social  compensation  or  disciplinary  action  to  reinforce 
or  correct  behavior  (e.g.,  “Command  leadership  encourages  reporting  safety 
discrepancies  without  the  fear  of  negative  repercussions.”). 

•  Quality  Assurance  -  policies  and  procedures  that  promote  high  quality 
performance  (e.g.,  “Quality  standards  in  my  command  are  clearly  stated  in  formal 
publications  and  procedural  guides.”). 

•  Risk  Management  -  how  the  organization  perceives  risk  and  takes  corrective 
action  (e.g.,  “My  command  takes  the  time  to  identify  and  assess  risks  associated 
with  its  flight  operations.”). 

•  Command  and  Control  -  policies,  procedures,  and  communication  processes  used 
to  mitigate  risk  (e.g.,  “Crew  rest  standards  are  enforced  in  my  command.”). 

On  the  basis  of  observations  and  interviews  with  maintainers,  the  MCAS  has  an  additional 
sixth  MOSE  called  “communication/functional  relationships.”  This  theme  is  concerned  with 
having  an  environment  in  which  information  is  freely  exchanged,  quality  assurance  is  seen  as  a 
positive  influence,  and  maintenance  workers  are  shielded  from  external  pressures  to  complete  a 
task  (Harris,  2000).  A  description  of  the  research  that  has  been  carried  out  using  the  MCAS  data 
will  be  described  first,  followed  by  studies  that  have  utilized  the  CSAS. 

1.  Maintenance  Climate  Assessment  Survey  (MCAS) 

A  considerable  amount  of  work  examining  the  psychometric  properties  of  the  MCAS  was 
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carried  out  by  Naval  Postgraduate  School  Masters’  students  in  the  late  1990s  and  early  2000s. 
Given  the  similarities  between  the  MCAS  and  CSAS,  and  the  lack  of  published  research  on  the 
CSAS  (see  below  for  a  discussion),  these  theses  will  be  briefly  described. 

The  MCAS  was  developed  by  Baker  (1998)  directly  from  the  CSAS.  He  carried  out 
Principal  Component  Analysis  (PCA)  on  268  responses  from  the  maintenance  personnel  of  three 
reserve  Naval  squadrons.  He  found  that  25  out  of  the  67  items  loaded  on  a  single  principle 
component.  However,  as  all  of  the  six  MOSEs  were  represented  in  this  principle  component,  he 
concluded  that  there  is  no  evidence  against  the  theoretical  underpinning  of  the  questionnaire.  As 
a  result  of  the  analysis,  Baker  (1998)  proposed  a  revision  of  the  questionnaire  consisting  of 
35  items. 

The  next  study,  carried  out  by  Oneto  (1999),  was  a  PCA  of  439  responses  collected  from 
maintainers  at  eight  reserve  squadrons.  Oneto  used  the  revised  survey  recommended  by  Baker 
(1998).  Again,  Oneto  (1999)  found  a  single  principle  component  that  explained  a  third  of  the 
variance.  As  this  principle  component  consisted  of  items  from  all  of  the  MOSEs,  he  also 
concluded  that  the  theoretical  model  was  sound. 

Goodrum  (1999)  assessed  the  1,000  responses  from  a  Naval  Air  Reserve  Fleet  Logistics 
Support  Wing.  Again,  following  PCA,  the  first  principle  component  explained  a  third  of  the 
variance,  with  the  six  items  that  loaded  the  highest  on  this  component  from  four  of  the 
six  MOSEs. 

Harris  (2000)  examined  the  responses  of  977  maintainers  at  a  Marine  Air  Wing.  Similar 
to  the  earlier  studies,  Harris  reported  a  single  principle  component  that  explained  a  third  of  the 
variance,  with  almost  all  of  the  items  from  the  questionnaire  loading  on  this  principle 
component.  Harris  then  used  the  six  MOSE  components  to  interpret  the  data,  and  found 
statistically  significant  differences  between  squadrons.  However,  he  did  not  find  a  statistically 
significant  relationship  between  safety  climate  and  aircraft-maintenance-related  incidents. 
Stanley  (2000),  using  the  same  dataset  as  Harris  (2000),  examined  the  relationship  between 
demographics  and  MCAS.  He  found  that  demographics  had  little  utility  in  predicting  the  scores 
of  a  given  unit. 

Hernandez  (2001)  examined  2,180  maintainer  responses  from  30  Naval  aviation  units 
using  the  online  and  paper  and  pencil  versions  of  the  test.  Similar  to  Harris  (2000),  she  did  not 
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find  that  demographic  data  correlated  with  the  MCAS  response.  The  results  of  a  PCA  of  the  data 
resulted  in  a  single  dominant  principle  component  that  explained  approximately  a  third  of  the 
variance.  Furthermore,  almost  all  of  the  questionnaire  items  loaded  on  this  principle  component. 
Hernandez  (2001)  did  not  find  a  significant  relationship  between  MCAS  score  and  aircraft- 
maintenance-related  incident  rate,  or  a  significant  difference  in  responses  based  upon  the  method 
of  completing  the  questionnaire. 

Most  recently,  Brittingham  (2006)  examined  the  MCAS  responses  from  126,058 
maintainers  collected  between  2000  and  2005.  After  completing  a  PCA,  she  found  that,  prior  to 
rotation,  one  principle  component  accounted  for  approximately  50%  of  the  variance.  She  states 
that  after  varimax  rotation,  a  second  principle  component  emerged.  The  first  principle 
component  consisted  of  items  concerned  with  overall  command  attention  to  safety,  and  the 
second  related  to  workload  and  the  availability  of  appropriate  resources.  However,  Brittingham 
(2006)  interprets  these  findings  very  differently  from  the  MCAS  studies  described  earlier.  As  the 
six  MOSE  components  were  not  identified  as  an  individual  factors  part  of  the  PCA  process, 
Brittingham  (2006)  states  that  “the  MCAS  was  found  to  be  an  inadequate  tool  with  questionable 
validity  for  gauging  maintenance  safety  climate”  (p.  31). 

It  could  be  argued  that  both  the  interpretation  of  Brittingham  (2006)  and  that  of  the 
earlier  studies  are  flawed,  due  to  the  lack  of  a  clear  understanding  of  the  methodology  that  was 
employed  to  identify  the  principle  components.  PCA  is  the  method  to  use  when  the  researcher  is 
attempting  to  reduce  a  large  number  of  variables  to  a  smaller  number  of  components  (Stevens, 
1996).  PCA  analyzes  variance  with  the  goal  of  extracting  the  maximum  variance  from  a  data  set 
with  a  few  orthogonal  (i.e.,  uncorrelated)  components  (Tabachnick  &  Fidell,  1996).  Since 
principle  component  scores  are  always  uncorrelated  by  construction,  unrotated  PCA  never 
accounts  for  correlations  between  the  presumed  factors  underlying  the  observations.  Furthermore 
principle  components  (or  their  coefficients)  are  never  chosen  with  reference  to  a  body  of  theory; 
they  always  arise  automatically  from  the  maximization  of  variance  explained. 

Another  related  issue,  which  may  have  accounted  for  the  large  proportion  of  items 
loading  on  a  single  principle  component,  is  the  large  proportion  of  respondents  responding 
positively  to  the  items.  To  illustrate,  Goodrum  (1999)  reported  that  all  questions  were  answered 
positively,  with  a  mean  range  of  between  3.17  and  4.37  (on  a  5-point  scale).  Hernandez  (2001) 
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reported  a  mean  range  between  3.18  and  4.15  for  the  items.  Therefore,  it  would  appear  that  there 
is  limited  variability  in  the  responses  to  the  items.  This  creates  problems  when  carrying  out  a 
PCA  because  if  all  of  the  items  have  a  similar  lack  of  variability,  then  the  PCA  will  tend  to 
identify  one  principle  component  with  a  large  number  of  items. 

The  other  problem  with  items  with  low  variability  is  that  they  are  not  useful  from  a 
discriminatory  perspective.  For  example,  Brittingham  (2006)  reported  that  for  item  7  “our 
command  climate  promotes  safe  maintenance,”  89%  of  respondents  agreed  or  strongly  agreed, 
and  only  6%  disagreed  or  strongly  disagreed.  Therefore,  this  item  is  not  useful  in  distinguishing 
between  high-  and  low-perfonning  groups  because  the  majority  of  participants  are  in  agreement. 
A  more  discriminatory  item  reported  in  the  Brittingham  (2006)  study  was  item  27  “day/night 
checks  have  equal  workloads  and  staffing  is  sufficient  on  each  shift.”  Although  it  could  be 
argued  that  this  item  is  asking  two  separate  questions  at  the  same  time,  at  least  there  is  some 
variance  in  response,  with  58%  agreeing  or  strongly  agreeing  and  34%  disagreeing  or  strongly 
disagreeing.  Therefore,  item  27  may  be  useful  in  discriminating  between  different  groups.  The 
danger  of  retaining  a  large  number  of  nondiscriminating  items  when  exploring  differences 
between  different  groups  of  respondents  is  that  the  discriminating  items  can  become  “washed 
out”  when  they  are  averaged  with  nondiscriminating  items.  Therefore,  the  use  of  PCA  with  a 
large  number  of  low  variance  items  may  account  for  finding  a  single  factor  on  which  the 
majority  of  MCAS  items  load. 

2.  The  Command  Safety  Assessment  Survey  (CSAS) 

Compared  to  the  MCAS,  there  has  been  much  less  research  published  examining  the 
CSAS.  An  unpublished  manuscript  of  an  exploratory  factor  analysis  of  1,254  surveys  resulted  in 
a  34-item,  3-factor  model  (Sengupta,  2000).  The  3-factor  model  was  also  found  to  be  an 
acceptable  fit  to  the  data  when  a  confirmatory  factor  analysis  was  carried  out.  No  attempt  was 
made  to  name  the  factors,  nor  was  there  any  discussion  of  the  results  in  the  manuscript.  In  a 
second  study,  Adamshick  (2007)  analyzed  the  data  of  every  Navy  and  Marine  Corps 
Strike-Fighter  aviator  that  completed  the  CSAS  from  2001  until  2005  (2,943  responses).  He 
carried  out  PCA  independently  for  the  items  that  make  up  each  of  the  five  theoretical  factors  of 
the  CSAS.  For  all  of  the  factors,  except  for  quality  assurance  and  reward  systems  (for  Naval 
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aviators  only),  it  was  found  that  a  two  or  more  factors  solution  resulted  in  a  better  fit  to  the 
theoretically-derived  factors  than  a  single  factor  model. 

Given  the  failure  of  both  of  these  studies  to  establish  a  factor  structure  that  is  consistent 
with  the  MOSE,  the  construct  validity  of  the  questionnaire  arguably  is  in  doubt.  Further,  the 
original  work  to  establish  the  factor  structure  was  carried  out  a  decade  ago.  The  safety  climate  of 
Naval  aviation  has  not  remained  stagnant  during  this  period.  A  number  of  safety  programs  have 
matured  and  become  more  widely  utilized  (e.g.,  crew  resource  management,  operational  risk 
management,  human  factors  councils/boards;  see  O’Connor  &  O’Dea,  2007  for  more  details). 
Therefore,  there  is  a  need  to  reexamine  the  factor  structure  and  assess  the  construct  validity  of 
the  CSAS. 

In  fact,  although  the  CSAS  was  used  unaltered  from  2000  until  2009,  the  content  of  the 
questionnaire  had  changed  recently.  The  MOSE  framework  model  was  abandoned  in  favor  of  a 
framework  that  is  loosely  based  upon  the  organizational  influence  and  supervisor  levels  of  the 
Human  Factors  Analysis  and  Classification  System  (HFACS;  Wiegmann  &  Shappell,  2003).  A 
total  of  31  items  from  the  original  CSAS  were  retained,  and  an  additional  16  items  were 
included.  The  rationale  behind  the  changing  of  the  theoretical  background  to  the  questionnaire, 
the  reasoning  behind  discarding  items,  and  how  the  new  items  were  selected  is  unknown  to  these 
authors.  Nevertheless,  this  revision  to  the  CSAS  does  not  negate  the  research  being  carried  out  to 
link  the  nine  years  of  CSAS  data  with  mishaps.  Rather,  this  research  effort  will  either  confirm 
the  changes  that  were  made  to  the  CSAS,  or  offer  guidance  as  to  how  the  questionnaire  can  be 
further  improved. 

Gaba,  Singer,  Sinaiko,  Bowen,  &  Ciavarelli  (2003)  compared  the  responses  of  health  care 
respondents  with  those  from  Naval  aviation.  Aviators  responded  to  CSAS  and  hospital  workers 
to  the  Patient  Safety  Cultures  in  Healthcare  Organizations  (PSYCHO)  survey.  Both  of  these 
instruments  have  partially  overlapping  items,  with  23  items  from  the  PSYCHO  adopted  directly 
from  the  CSAS.  The  survey  included  employees  from  15  hospitals  and  Naval  aviators  from  226 
squadrons.  For  each  question  a  “problematic  response”  was  defined  that  suggested  a  lack  of  or 
antithesis  to  safety  climate  (Gaba  et  ah,  2003).  Overall,  the  problematic  response  rate  for  hospital 
workers  was  up  to  12  times  greater  than  that  among  aviators  on  certain  items.  These  findings 
were  true  both  for  the  aggregate  of  all  health  care  respondents  and,  even  more  strikingly,  for 
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respondents  from  particularly  hazardous  health  care  arenas  (e.g.,  emergency  rooms  and  critical 
care)  the  number  of  problematic  responses  were  16  times  greater  than  among  aviators. 

However,  the  study  did  reveal  a  few  similarities  between  hospital  personnel  and  Naval 
aviators  regarding  specific  safety  climate  features  covered  by  the  matched  questions.  In  both 
sectors,  respondents  were  highly  uniform  in  their  belief  that  their  institution  is  committed  to  and 
has  a  good  reputation  for  safety.  They  both  expressed  concern  about  the  level  of  resources 
provided  for  them  to  accomplish  their  jobs,  although  health  care  workers  were  even  more 
concerned  than  aviators  about  the  effect  on  safety  of  a  loss  of  experienced  personnel. 
Nonetheless,  for  most  questions  across  all  aspects  of  safety  climate,  there  were  low  rates  of 
problematic  response  among  Naval  aviators  (generally  under  10%),  but  a  much  higher  rate 
among  health  care  workers,  by  a  factor  of  three  or  more.  Thus,  the  overall  pattern  of  results 
suggests  that  the  military  safety  climate  is  quite  high  compared  to  other  HROs. 

Desai  et  al.  (2006)  measured  the  relationship  between  recent  accidents  and  perceptions  of 
safety  climate,  as  measured  by  the  CSAS,  on  a  large,  cross-sectional  sample  of  respondents  in 
several  Naval  aviation  squadrons.  The  notion  was  to  understand  potential  cognitive  and 
behavioral  changes  following  accidents.  They  hypothesized  that  safety  climate  would  improve 
after  an  accident  occurred  because  actual  changes  in  safety  climate  occurred,  or  cognitive  bias 
(fundamental  attribution  error)  occurs  in  which  people  are  more  likely  to  blame  situational 
factors  rather  than  people. 

The  study  used  the  6,361  responses  from  147  Naval  squadrons  taking  the  online  CSAS 
between  July  2000  and  December  2001.  Aviation  mishap  infonnation  was  collected  from  the 
U.S.  Naval  Safety  Centre  (the  number  of  mishaps  used  was  not  reported).  These  accidents  are 
measured  by  their  intensity  and  are  divided  into  Class  A,  Class  B,  and  Class  C  mishaps.  At  the 
time  of  the  research,  the  definition  of  a  Class  A  mishap  was  damage  of  $  1  million  or  more,  or  an 
injury  or  occupational  illness  resulting  in  a  fatality  or  pennanent  total  disability.  Class  B  mishaps 
involve  a  total  mishap  cost  of  $200,000  or  more,  but  less  than  $1  million,  or  an  injury  or 
occupational  illness  that  results  in  permanent  partial  disability  or  for  which  three  or  more  persons 
are  hospitalized.  Class  C  mishaps  are  accidents  in  which  the  total  cost  of  reportable  material 
property  damage  is  $10,000  or  more,  but  less  than  $200,000,  a  nonfatal  injury  that  causes  any 
loss  of  time  from  work  beyond  the  day  or  shift  on  which  it  occurred,  or  a  nonfatal  illness  or 
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disease  that  causes  loss  of  time  from  work  because  of  disability  (Chief  of  Naval 
Operations,  2001). 

The  dependent  variable  was  a  safety  climate  perception  construct  developed  by 
aggregating  each  individual’s  responses  to  the  CSAS.  Six  independent  variables  were 
constructed  to  measure  accidents  prior  to  survey  administration.  These  mishap  variables  were 
recorded  at  the  squadron  group  level  of  analysis.  All  individuals  within  the  squadron  received  the 
squadron  value  for  these  mishap  variables  for  the  present  analysis. 

Desai  et  al.  (2006)  regressed  the  safety  climate  construct  on  several  indicator  variables 
tracking  the  occurrence  of  accidents,  grouped  by  their  severity,  in  periods  roughly  one  year  prior 
to  survey  measurement  and  two  years  prior  to  survey  measurement.  Analysis  indicated  positive 
associations  between  minor  or  intermediately  severe  accidents  and  future  safety  climate  scores, 
although  no  effect  was  found  for  major  accidents.  These  findings  suggest  a  generally  positive 
association  between  minor  or  intermediately  severe  accidents  and  perceived  safety  climate.  This 
study  suffers  in  that  only  limited  information  was  obtained  on  the  mishaps.  Also,  although  the 
number  of  mishaps  that  occurred  during  the  period  of  study  were  not  reported,  the  number  was 
likely  to  be  fairly  low.  Finally,  the  rationale  that  the  safety  climate  will  improve  after  a  mishap 
may  be  flawed.  If  the  squadron  personnel  believe  that  the  causes  of  the  mishap  have  not  been 
addressed,  it  may  be  that  the  safety  climate  may  go  down,  rather  than  improve,  as  suggested  by 
Desai  et  al. 

One  unpublished  study  investigated  whether  the  responses  to  the  CSAS  can  predict 
aviation  mishap  rate.  After  some  earlier  encouraging  analysis  in  support  of  the  predictive  validity 
of  the  CSAS,  Schimpf  and  Figlock  (2006)  took  the  average  (it  is  assumed  that  average  refers  to 
mean,  although  this  is  not  stated)  of  the  nine  items  from  the  risk  assessment  MOSE  (the  rationale 
for  the  focus  on  this  particular  MOSE  was  not  provided),  as  well  as  the  overall  average  of  the 
61 -item  CSAS  for  each  respondent  from  August  2000  until  October  2004.  They  divided  the 
squadrons  into  quartiles  based  upon  the  average  scores.  They  then  counted  the  number  of 
squadrons  that  had  experienced  a  Class  A  mishap  within  12  months,  18  months,  and  24  months 
after  taking  the  survey  within  each  quartile  (no  explanation  was  provided  for  how  squadrons  that 
had  completed  the  questionnaire  on  multiple  occasions  within  this  time  period,  or  squadrons  that 
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had  multiple  Class  A  mishaps,  were  handled).  The  data  from  this  analysis  are  summarized  in 
Figures  1  and  2. 


(Surveys  conducted  from  10AUG00-10OCT04) 
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Figure  1.  Class  A  mishaps  within  12,  18,  and  24  months  after  completing  the  CSAS  (Quartiles  by 

overall  CSAS  average;  from  Schimpf  &  Figlock,  2006). 


Figure  2.  Class  A  mishaps  within  12,  18,  and  24  months  after  completing  the  CSAS  (Quartiles  by 

risk  management  average;  from  Schimpf  &  Figlock,  2006). 

These  findings  are  encouraging  for  the  predictive  validity  of  the  questionnaire.  However, 
collapsing  the  questionnaire  data  to  the  extent  that  was  done  in  this  study  is  a  coarse  method  to 
examine  whether  the  CSAS  is  a  useful  predictor  of  mishap  probability.  Reducing  a  sample  size 
of  some  3,355,000  questionnaire  responses  (i.e.,  approximately  55,000  responses  to  61  items)  to 
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four  data  points  (i.e.,  quartiles)  would  seem  to  be  a  very  wasteful  use  of  data,  and  will  result  in  a 
very  large  restriction  in  variability.  Moreover,  if  items  for  which  there  is  little  variability  are 
included  to  calculate  the  mean,  as  discussed  above  with  reference  to  the  MCAS,  those  items  for 
which  there  are  variance  may  be  washed  out.  It  would  also  have  been  of  interest  to  have  seen  the 
mean  and  standard  deviation  of  the  quartile  scores  (these  were  not  reported).  The  final  concern  is 
that  the  rationale  for  choosing  the  risk  management  scale  is  not  provided.  It  would  have  been 
interesting  to  know  what  would  have  been  produced  using  the  same  methods,  but  with  the  other 
scales.  Schimpf  and  Figlock  (2006)  also  concluded  that  MCAS  item  34  (lam  provided  adequate 
resources,  time,  personnel  to  accomplish  my  job.)  was  also  a  good  indicator  of  Class  A  mishap 
risk,  using  the  same  method  as  described  above  (again,  the  reason  for  focusing  specifically  upon 
only  this  item  is  not  delineated). 

In  addition  to  the  PCA  described  above,  Adamshick  (2007)  also  used  the  CSAS  and 
MCAS  to  assess  the  relationship  between  leadership  interventions  and  a  respondent’s  safety 
climate  assessment.  Most  pertinent  for  this  review  were  results  regarding  CSAS  item  42  (my 
command  provides  a  positive  command  climate  that  promotes  safe  flight  operations )  and  MCAS 
item  7  (our  command  climate  promotes  safe  maintenance).  For  CSAS  item  42,  the  following 
rank/demographic  differences  emerged  among  Navy  and  Marine  respondents:  senior  officers 
reported  significantly  higher  scores  than  junior  officers;  among  pilots,  those  with  more  than 
2,000  flight  hours  reported  significantly  higher  scores  than  those  with  fewer  hours,  especially 
those  who  had  between  500  and  1,000  flight  hours.  In  addition,  Navy  department  heads  reported 
significantly  higher  scores  than  nondepartment  heads  and,  among  Marines,  differences  emerged 
between  reservists  in  the  following  order:  driller  reserves  had  higher  scores  than  active  reserves, 
who,  in  turn,  reported  higher  scores  than  regular  status  respondents. 

For  MCAS  item  7,  rank  differences  occurred  among  Naval  respondents,  in  which  officers 
tended  to  report  higher  scores  than  enlisted;  among  enlisted,  the  higher  the  rank,  the  higher  the 
score.  Work  frame  differences  also  emerged,  in  which  respondents  in  maintenance  control  had 
the  highest  scores,  whereas  avionics  reported  significantly  lower  scores  than  most  other  work 
frames.  As  would  be  expected,  night  shift  respondents  reported  lower  scores  than  day 
shift  respondents. 
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Adamshick  (2007)  suggests  a  variety  of  reasons  for  these  demographic  differences.  For 
example,  rank  differences  may  be  due  to  senior  officers’  bias  in  rating  programs  for  which  they 
are  responsible.  Junior  enlisted  may  be  more  frustrated  than  senior  enlisted  due  to  increased 
responsibility,  yet  not  commensurate  rank  increase.  Regarding  the  response  difference  by 
number  of  total  flight  hours,  it  may  be  that  those  pilots  with  500-1,000  hours  are  no  longer 
novice  pilots  that  find  flying  challenging  and,  at  the  same  time,  have  started  to  have  some 
authority.  Adamshick  also  points  out  that  greater  flight  hours  are  positively  correlated  with  rank 
and  authority. 

Adamshick’ s  (2007)  results  also  indicate  that  perceived  leadership  factors  positively 
associated  with  safety  climate  differ  between  officers  and  enlisted.  For  officers,  four 
factors  emerged: 

1.  use  Human  Factors  Boards  (a  regular  proactive,  informal  review  of  all  officer  and 
enlisted  aircrew;  see  O’Connor  &  O’Dea,  2007  for  more  details); 

2.  leadership  that  encourages  and  enables  individuals  to  report  unsafe  behaviors; 

3.  leadership  that  successfully  communicates  safety  goals  to  personnel;  and 

4.  leadership  that  reacts  to  unexpected  changes. 

For  enlisted  respondents,  three  leadership  factors  were  positively  associated  with 
safety  climate: 

1 .  leadership  adequately  reviews  and  updates  safety  procedures; 

2.  leadership  does  not  tolerate  unprofessional  behavior;  and 

3.  leadership  uses  comprehensive  and  effective  safety  education  and  training 
programs. 

Thus,  in  comparison  to  the  Schimp  and  Figlock  (2006)  report,  Adamshick’ s  results  suggest  that  a 
finer-grained  analysis  of  the  CSAS  and  MCSAS  is  merited. 

G.  CONCLUSION 

It  is  argued  that  safety  culture  surveys  can  retrieve  information  that  is  not  accessible 
through  other  more  traditional  methods  of  analysis,  such  as  audits  and  risk  assessments.  Bailey 
and  Petersen  (1989)  concluded  that  the  effectiveness  of  safety  programs  cannot  be  measured  by 
the  more  traditional  procedural-engineering  criteria  popularly  thought  to  be  factors  in  successful 

programs.  They  argue  that  a  better  measure  of  safety  program  effectiveness  is  the  response  from 
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the  entire  organization  to  questions  about  the  quality  of  the  management  systems  that  have  an 
effect  on  human  behavior  relating  to  safety.  They  further  concluded  that  perceptions  surveys  can 
effectively  identify  the  strengths  and  weaknesses  of  a  safety  system’s  elements.  However,  for  a 
safety  climate  survey  to  be  useful,  it  must  have  construct  and  discriminate  validity. 

It  is  suggested  that  a  comprehensive  assessment  of  the  validity  of  the  CSAS  is  long 
overdue.  The  construct  validity  of  the  questionnaire  has  never  been  established,  and  there  is  only 
weak  evidence  supporting  the  discriminate  validity  of  the  tool.  There  is  no  specific  proof  that  the 
CSAS  is  not  identifying  “at  risk”  squadrons.  However,  there  is  also  no  strong  evidence  that  it  is 
supplying  helpful  infonnation  to  leadership.  In  the  absence  of  a  valid  tool,  time  and  money  is 
being  wasted  administrating  the  survey.  However,  most  importantly,  the  opportunity  of 
preventing  mishaps  by  providing  useful  feedback  to  leadership  is  being  missed. 
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